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BACKGROUND OF THE INVENTION 

Field of the Invention 

[0001] The present invention relates to imaging systems. More particularly, it 

relates to imaging devices having a fixed focal length. 

Related Art 

(0002] Many imaging systems have an imager that receives energy and converts 

the received energy into a form of data known as pixels. This pixel data can then 
be stored by these imaging systems for subsequent retrieval and processing and/or 
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transmitted to a remote receiving station for storage and processing. Pixel data 
can be processed using known signal processing techniques. Improving the signal 
quality and/or reducing the data output rates of these imaging systems is highly 
desirable. 

[0003] Imaging systems having a fixed focal length imager are designed to image 

objects or areas with a minimum image resolution while operating at a maximum 
design distance from the objects or areas to be imaged. If these imaging systems 
are used to image objects at a distance in excess of their maximum design 
distance, they will not be able to achieve their designed minimum image 
resolution. These imaging systems are often used, however, to image objects or 
areas that are nearer then their maximum design distance. For example, certain 
airborne imaging systems are designed to image ground objects with a minimum 
image resolution while being carried aboard an- aircraft flying at a specified 
maximum altitude. But, since the operational, envelope of an aircraft covers a 
range of flying altitudes, there are often times when these airborne imaging 
systems are used while being carried aboard an aircraft flying at an altitude less 
than the specified maximum design altitude for the imaging system. When this 
occurs, these imaging systems will have an achievable image resolution in excess 
of their designed minimum image resolution and a less than optimal signal-to-noise 
ratio. Moreover, when this occurs, there is an excess of pixel data that must be 
stored by these imaging systems for subsequent retrieval and processing and/or 
transmitted to a remote receiving station for storage and processing. This is a less 
than desirable situation when high speed data transmission or processing is 
required. 

[0004] What is needed is a means for improving the signal quality and/or reducing 

the data output rates of imaging systems having an achievable image resolution in 
excess of their designed minimum image resolution. 



SUMMARY OF THE INVENTION 



[0005] The present invention is directed to enhancing the performance of a fixed 

focal length imaging device. An input group of pixels obtained using an imager 
is converted into an output group of pixels having an equal or lower pixel count. 
During the conversion process, the average signal-to-noise ratio of the output 
pixels may be improved. 

[0006] First and second input pixels having an initial intensity value are received 

and temporarily stored in a memory, A first intermediate intensity value is formed 
from the first input pixel based on the initial intensity value of the first input pixel 
and the value of a first weighting factor. Similarly, a second intermediate intensity 
value is formed from the second input pixel based on the initial intensity value of 
the second input pixel and the value of a second weighting factor.. At least one 
. . intermediate intensity value formed from the first input pixel and at least one 
intermediate intensity value formed from the second input pixel are combined to 
form at least one output pixel 

[0007] It is a feature of the present invention that the distance between the imager ! 

and an object can be used to select the number of output pixels formed. 

[0008] It is another feature of the present invention that a change in distance 

between the imager and an object can be used to dynamically adjust the number 
of output pixels formed. 

[0009] A further feature of the invention allows a variable resolution image to be 

converted to a fixed resolution image. 

[0010] Further features and advantages of the present invention, as well as the 

structure and operation of various embodiments of the present invention, are 
described in detail below with reference to the accompanying drawings. 

[0011] 

BRIEF DESCRIPTION OF THE FIGURES 

[0012] The present invention is described with reference to the accompanying 

figures. In the figures, like reference numbers indicate identical or functionally 



similar elements. Additionally, the leftmost digit or digits of a reference number 
identify the figure in which the reference number first appears. The accompanying 
figures, which are incorporated herein and form part of the specification, illustrate 
the present invention and, together with the description, further serve to explain 
the principles of the invention and to enable a person skilled in the relevant art to 
make and use the invention. 
[0013] FIG. 1 illustrates the relationship between a pixel length (P) and a ground 

sample distance (GSD) of an airborne imaging system according to the present 
invention; 

[0014] FIG. 2 is a flowchart of the method of the present invention; 

[0015] FIG. 3 illustrates sampling locations (fl) obtained from imaging at an 

arbitrary altitude and the interpolated sample points (f2)used to maintain constant 
.image resolution; 

[0016] FIG.* 4 illustrates a. continuous intensity waveform imposed upon the 

. sampling points of FIG. 3 ;. -.1 .a ^ . : ■* . ; ' . . . . . 
[0017] FIG. 5 illustrates the system of the present invention; 

[0018] FIG. 6 illustrates an example interpolation function; 

[001 9] FIG. 7 illustrates an example waveform showing the relationship between 

initial sampled values of an imaging system and sampled values after interpolation; 

[0020] FIGs. 8A-C illustrate how groups of output pixels are formed from groups 

of input pixels; and 

[0021] FIG. 9 is a flowchart of a method for transforming image resolution. 

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS 

[0022] The present invention provides a method and system for enhancing the 

performance of imaging devices. A group of input pixels obtained by an imager 
having a fixed focal length is converted into a group of output pixels having an 
equal or lower count while maintaining image resolution. During this conversion 
process, the average signal-to-noise ratio (SNR) of the pixels maybe improved. 
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A feature of the present invention allows a distance between the imager and an 
object to be used to select the number of output pixels formed. Another feature 
of the present invention allows a change in distance between the imager and the 
object to be used to dynamically adjust the number of output pixels formed. A 
further feature of the invention allows a variable resolution image to be converted 
to a fixed resolution image. 

[0023] As will be apparent from the detailed description herein, the present 

invention is particularly well adapted for use with an airborne imaging system 
where the distance between the imager and an object being imaged ranges 
typically between about 5,000 feet and about 65,000 feet. The invention is not 
limited to being used only with airborne imaging systems, however. 

(0024] The detailed description of the present invention that follows begins with 

a terminology subsection that defines terms used to describe the present invention. 
.. This subsection is followed by a detailed description of a preferred application of 
the present invention. A preferred application of the invention is presented at the 
beginning of this section in order to aid the reader's understanding of the present 
invention. Finally, this section concludes by describing in detail both system 
embodiments and method embodiments of the present invention. 

Terminology 

[0025] The following terms are defined so that they may be used to describe 

embodiments of the present invention. As used herein: 
[0026] "Image" or "scene" means an array of pixels. 

[0027] "Imager" means that portion of an imaging system that converts 

electromagnetic energy into pixels. An imager can comprise, for example, a 
photodetector that is responsive to particular wavelengths of electromagnetic 
energy and any lenses or mirrors used to focus electromagnetic energy onto the 
photodetector. 

[0028] "Pan scan imager" means an imager that uses a linear photodetector array 

and that scans an image by translating or rotating a line array detector across a 
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scene to build up an image one line at a time. The direction of motion of the array 
is normally orthogonal to the direction of travel of an aircraft. A slowly rotating 
mirror may be placed in the optical path between the scene and the photodetector 
array to avoid smearing and compensate for the forward motion of the aircraft. 

[0029] "Pixel" means a data structure used to represent a picture element. Any 

type of pixel data format can be used with the present invention. 

[0030] "Staring imager" means an imager with a two dimensional photosensitive 

array that records an image on all pixels simultaneously. A staring imager does 
not rely on scanning motions to buildup an image one line at a time. 

[003 1 ] "Tactical reconnaissance imaging" means imaging performed by an aircraft 

operating below an altitude of about 65,000 feet and flying below an airspeed of 
about 800 knots. 

Example Application of the Present Invention 

[0032] The present invention is particularly well adapted for use with an airborne 

imaging system. As will be described herein, the present invention can be used to 
enhance the performance of either a pan scan imager or a staring imager of a type 
typically used for tactical reconnaissance imaging. The present invention is 
described herein with reference to an airborne imaging system because that is a 
preferred use of the invention by the inventor. However, as will be apparent to 
a person skilled in the relevant art given the description of the invention herein, the 
present invention is not limited to being used only with an airborne imaging 
system. The present invention can be used to enhance the performance of any 
imaging device having a fixed focal length. The invention can also be used to 
convert a variable resolution image to a fixed resolution image. 

[0033] As would be known to a person skilled in the relevant art, the resolution 

of an airborne imaging system depends on the focal length and pixel size of the 
system's imager, as well as the range from the imager to the object being imaged. 
These relationships are illustrated in FIG. 1. 
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{0034] FIG. 1 depicts an aircraft 1 00 performing tactical reconnaissance imaging. 

Aircraft 100 has an onboard imaging device (not shown), which is typically 
contained in the nose of aircraft 100 or in a pod mounted to the underside of the 
aircraft. The onboard imaging device has an imager (not shown) having a fixed 
focal length "f" and an instantaneous field-of-view "P." The resolution of an 
imaging device can be described in terms of instantaneous field-of-view and 
ground sample distance (GSD). Instantaneous field-of-view refers to that portion 
(angular extent) of a ground object or area falling on a single pixel. GSD refers 
to a measured ground distance that corresponds to a pitch between adjacent pixels 
of an image. Instantaneous field-of-view is typically measured in radians, and 
GSD is typically measured in meters or feet. Small instantaneous field-of-view 
values or GSD values represent high image resolution. 

[0035] As aircraft 100 changes altitude, an object on the ground appears to 

change size to an observer onboard aircraft 100 or to an imaging device having 
a fixed focal length onboard aircraft 1 00. For example, if an object on the ground 
covers one pixel of the imager (not shown) onboard aircraft 1 00 when aircraft 1 00 
is flying at an altitude of 1 6,000 feet, the same object will cover two pixels of the 
imager when aircraft 100 is flying at an altitude of 8,000 feet. This property of an 
imager having a fixed focal length can be explained using EQ. 1. 

P GSD 

— = EQ.l 

f R 

where: P is the length of a pixel of an imager; 
/is a fixed focal length of the imager; 
GSD is ground sample distance; and 

R is an approximate range from the imager to a ground object. 



[0036] Looking at FIG. 1, it will be apparent that the equality of EQ. 1 is a 

correct statement ofbasic trigonometry principles, assuming that the instantaneous 

P 

field of view (IFOV), — , is a small angle. 

[0037] In Table 1, below, EQ. 1 has been rewritten to show that the number of 

pixels covered by an object changes with a change in the altitude of aircraft 100. 
As show in Table 1 , the number of pixels covered (in one dimension) by an object 
doubles as the altitude of aircraft 1 00 is lowered by one-half For a slant range of 
16,000 feet, a focal length of 3.84 inches, and a pixel length of 20 /^m, the GSD 
is one meter. If a footprint of one meter is viewed from 8,000 feet, it covers a 
length of 40 /xm on the focal plane of the imager, i.e., two of the original 20 jxm 
pixels. 



P = (/• GSD)/R 


P = 20 urn 


P = 40 urn 


/= 3.84 ins 


/=3.84 ins 


GSD = 1 m 


GSD = lm 


R= 16,000 ft 


R = 8,000 ft 



Table 1 



For a pan scan imager with fixed focal length, f, designed to achieve a one 
meter GSD at 16,000 feet, the resolution of the imager is twice as good as 
necessary at 8,000 feet. The additional resolution of the imager can be traded 
therefore for additional signal by adding together the signals from each group of 
two adjoining pixels when flying at an altitude of 8,000 feet, achieving the same 
resolution as at 16,000 feet, but with twice the signal level. 

The pixels on a detector array may be thought of as image sample points 
with a sampling period equal to the distance between pixel centers. Thus, the 
spatial frequency of the pixels is 1 /period. At an altitude of 16,000 feet, one 
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20 /im pixel just covers a GSD of one meter, and the sampling frequency 
corresponding to 20 jum pixels is 50 lines/mm. The impact of changing the 
altitude of aircraft 100 from 16,000 feet to 8,000 feet is that a GSD of one meter 
now covers two 20 fxm pixels, thereby permitting the sampling frequency of the 
pixels to be reduced to one-half the prior sampling frequency used at the 16,000 
foot altitude (i.e., 25 lines/mm) while still providing the same image resolution 
(e.g., one pixel for each GSD). 
[0040] The present invention uses the impact that changing aircraft altitude has 

on image resolution to enhance the performance of imaging devices having a fixed 
focal length. The system and method described herein teach one skilled in the 
relevant art how to convert a series or group of contiguous pixels obtained by an 
imager having a fixed focal length into a second series or group of pixels of equal 
or lesser count. The size of the input and output groups of pixels is based upon 
a repeat interval at which original sampling positions (pixels) coincide with 
sampling positions (pixels) interpolated according to the present invention. In this 
conversion process, the average signal-to-noise ratio (SNR) of the interpolated 
pixels is enhanced. 

Example System Embodiment of the Present Invention 

[0041] FIG. 5 illustrates a system 500 for enhancing the performance of a fixed 

focal length imaging device. System 500 converts a group of five input pixels into 
a group of four output pixels. System 500 comprises a pixel receiving module 
510, a pixel weighting module 520, and a pixel combining module 580. 

[0042] Pixel receiving module 5 1 0 is typically coupled to the output of an imager 

(not shown). It may be coupled to a digital filter, however. Digital filters may be 
used to assure that no frequencies above those desired are present in the input 
pixels received by pixel receiving module 510. This is done to improve signal 
quality. Pixel receiving module 5 10 is shown as receiving five input pixels {Y,, 
Y 2 , Y 3 , Y 4 , Y 5 } from an input stream of pixels (not shown). Pixel receiving 
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module 5 1 0 can receive and temporarily store more or less than five pixels. Pixel 
receiving module 5 1 0 can be a buffer. 
[0043] Pixel weighting module 520 is coupled to pixel receiving module 510. 

Module 520 comprises five subsets 530, 540, 550, 560, and 570 of four weighting 
factors or weighting cells. For example, subset 530 comprises weighting cells 
532, 534, 536, and 538. The number of weighing cells in each subset is dependent 
on the number of output pixels formed by system 500. Each weighting factor or 
weighting cell of pixel weighting module 520 is used to form an intermediate 
weighting value from an input pixel stored in pixel receiving module 510. 
Referring to EQ. 2 below, the value for each weighting factor or weighting cell 

";f is selected in accordance with an interpolation function. 

[0044] System 500 can also convert a group of five input pixels into a group of 

= := fewer than four output pixels, for example, a group of three, two, or one pixel (s). 

r: The values for each weighting cell are adjusted depending on the number of 
output pixels to be produced and the particular output pixels to be discarded or 

0 . ignored. For example, if a group of five input pixels is to be converted into two 
r[ output pixels, the value for each weighting cell W 53 , W 54 , W 43 , W 44 , W 33 , W 34 , W 23 , 
J 1 ] W 24 , W I3) and W I4 is set to zero, and the output pixels Z 3 and Z 4 are discarded or 

1 a ignored. Hardware and/or software means can be used for adjusting the values 

of the weighting cells and for selecting the particular output pixels to be discarded 
or ignored. 

[0045] Pixel weighting module 520 can be built from commercially available 

electronics hardware. For example, pixel weighting module 520 can be built 
using known discrete small scale integration chips (e.g., digital and/or analog 
multiplier chips, adder chips, and memory chips) or very large scale integration 
chips (e.g., an embedded microprocessor). Pixel weighting module 520 can also 
be built, for example, using a microcontroller or a digital signal processing array. 
A microcontroller such as the MSC 51, MSC 151, or MSC 251 (or the 8051 or 
8052) available from Intel Corporation, Santa Clara, California, can be used to 
implement pixel weighting module 520. In preferred embodiments of the present 
invention, the values for each weighting cell or factor can be adjusted (e.g., the 
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weighting factors might be calculated and stored in a lookup table in memory, and 
retrieved from the lookup table as needed). 

[0046] Pixel combining module 580 is coupled to pixel weighting module 520. 

As shown in FIG. 5, pixel combining module 580 comprises four addition modules 
582. Each addition module 582 combines the output of one weighting cell from 
each subset of weighting cells to form a single output or pixel. 

[0047] Pixel combining module 580 is shown coupled to a buffer 590. Buffer 590 

receives outputs from pixel combining module 580 and temporarily stores the 
received outputs. The received outputs or pixels stored in buffer 590 can be, for 
example, transmitted to a remote receiving station for processing and storage, or 
they can be stored in a memory (not shown) for subsequent retrieval and 
processing. 

[0048] Other system embodiments of the present invention are possible, which 

. convert a preselected number of input pixels (e.g., 8, 16, 32, 64) to a selected 
. number of output pixels. In all embodiments, a distance between the imager and 
an object to be imaged is used to select the number of output pixels formed, as 
described below. In some embodiments, a change in distance between the imager 
and an object to be imaged is used to dynamically adjust the number of output 
pixels formed. 

Example Method Embodiment of the Present Invention 

[0049] FIG. 2 illustrates a flowchart of a method for enhancing the performance 

of an imaging device having a fixed focal length according to the present 
invention. This method can be implemented using system embodiments of the 
present invention (e.g., system 500), and it is described with reference to the 
features illustrated in FIG. 5. As described below, the method comprises six 
steps, of which three steps are optional steps. 

[0050] In step 202, a group of input pixels formed by an imager is received and 

temporarily stored in a memory, e.g., buffer 510. The group of input pixels 
received in step 202 is variable as described below (i.e., it is not necessarily five 
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pixels as shown in FIG. 5). Typically, the received group of input pixels will be 
a contiguous series of pixels obtained by the imager. 
[0051] In optional step 204, the pixels received in step 202 are filtered in order 

to prevent aliasing. Step 204 can be performed either before step 202 or after step 
202. There are many lowpass filters and/or filtering techniques that can be applied 
to the pixels received in step 202 in order to prevent aliasing. Two such filters are 
the N^-order Butterworth filter and the Chebyshev filter. See Gerald C Holst, 
Sampling, Aliasing, and Data Fidelity forElectronic Imaging Systems, 
Communications, and Data Acquisition 239-42 (JDC Publishing, Winter 
Park, FL (1998)) (SPIE Press Series, No. 55). Given the description of the 
present invention herein, a person skilled in the relevant art will be able to 
"| determine whether optional step 204 should be implemented in a given 

embodiment of the present invention. 
:J- S [0052] . ■ Steps 206 and 208 of the method are best described together. In step 206, 
" s . . . at least one intermediate intensity value for. each pixel received in step 202 is 
;0 formed. In step 208, the intermediate intensity values formed in step 206 are 

<f\i 

combined to form a group of output pixels. How to implement steps 206 and 208 
^ will now be described in detail below with reference to the inventor's preferred 

|=s= application of the present invention. 

[0053] FIG. 3 illustrates two sets of sampling points, one set labeled by f, and one 

set labeled by f 2 . .If the set labeled f, is assigned to the sample points (pixels) 
obtained by an imager onboard aircraft 100 at an altitude of 16,000 feet, the 
number of sample points (pixels), labeled f 2 , needed to achieve the same resolution 
at an altitude lower than 1 6,000 feet is less than the number of fj samples. As 
described above, only half the number of sampling points is needed at an altitude 
of 8,000 feet. At altitudes between 8,000 feet and 1 6,000 feet, more than half the 
sampling points are needed, but the number needed is still less than the number of 
fi sampling points. 

[0054] Notice in FIG. 3 that two of the fj and f 2 sampling points occur at the same 

location. The series of sampling points 300 shown in FIG. 3 is cyclical, thus if the 
series of sampling points f, and f 2 shown were extended, the number of coincident 
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sampling locations would increase. This coincidence of sampling locations is used 
herein to describe the present invention. 

[0055] The image of a scene laid out on a focal plane of a staring imager contains 

a continuous set of two-dimensional intensity values. For simplicity, the 
description of the present invention herein focuses mainly on a one-dimensional 
representation of the image (e.g., as can be obtained using a pan scan imager). 
This description of the present invention can be extended to two dimensions, 
however, by simply considering the one-dimensional discussion herein as referring 
to a single line in the array. Thus, the description of the present invention herein 
can be applied to both a staring imager and a pan scan imager. 

[0056] The spatial variation of a representative image intensity signal 400 is 

shown as a curved line 402 in FIG. 4. The sampling points f, and f 2 shown in 
FIG. 3 are also shown in FIG. 4. The fj samples points (corresponding to an 
altitude of 16,000 feet) are assigned amplitudes y s (the index "i" enumerating all 
. the. individual pixels). The f 2 sampling points are assigned amplitudes Zj. 

[0057] As described above, the number of f 2 samples is less than or equal to the 

number of f, samples over a given spatial interval. The f, sampling points 
correspond to the (fixed) pixels in the focal plane array of an imager. The 
intensity levels, y ( , are the signal levels at each of the pixels. The f 2 sampling 
points correspond to the sampling points required to maintain constant resolution 
as the altitude of aircraft 100 changes. The location of the f 2 samples shown in 
FIG. 4 varies with altitude. 

[0058] The number of output pixels formed in step 208 is related to the number 

of input pixels received in step 202 by a ratio of small numbers. The spatial 
location of certain output pixels will coincide with the spatial location of certain 
input pixels at regular intervals, as illustrated in FIGs. 3 and 4. By taking 
advantage of these spacial coincidences, the system of the present invention can 
reduce hardware requirements. 

[0059] In order to form intermediate intensity values and combine these values to 

form a group of output pixels, as recited in steps 206 and 208, it is desirable to 
use the y s pixel intensities and the locations of the f 2 sampling points to determine 
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the Zj intensity levels. In general, each z } is a function of all the y ; pixels. 
However, a simplification may be used in implementing the method so that each 
Zj is expressed as a function of only a limited number of y/s. This simplification 
takes advantage of the coincidence of certain f, sampling points with certain f 2 
sampling points. 

[0060] As shown in FIG. 3, the f, and f 2 sampling points will coincide at regular 

intervals whenever the frequencies of the two sets of sampling points are a 
submultiple of each other. Table 2, below, illustrates the relationship between 
spatial frequencies at different altitudes for an imager capable of achieving a GSD 
of one meter at 16,000 feet. Table 2 lists two sets of sampling frequencies for the 
imager, which are submultiples of each other. The altitudes at which submultiples 
of frequencies occur, and the number of samples (pixels) obtained by the example 
imager between an occurrence of coincidences, is also shown. Although Table 2 
lists altitudes ranging between only 1 6,000 feet and 8,000 feet, it should be noted 
that the present invention is not limited to these altitudes. 

[0061 ] As shown in Table 2, the altitudes at which spatial frequencies are (small) 

submultiples of each other maybe designed according to the present invention to 
occur at increments of 1 ,000 feet. At 1 5,000 feet, for example, a spatial frequency 
in the ratio of 15:16 with the spatial frequency at 16,000 feet can achieve a one 
meter GSD. At 1 5,000 feet, therefore, the present invention can be used with the 
example imager to replace 1 6 pixels with 1 5 pixels while still maintaining the same 
image resolution that the example imager would have as at 1 6,000 feet. 
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Relationship Between Spatial Frequencies At Different Altitudes 



Altitude 


Freq. Needed To 
Maintain A 
Constant GSD 
(Lines/mm) 


Ratio Of Freq. At 
16,000 Feet To Freq. 
At A Given Altitude 


No. Of Fl Lines 
Between 
Coincidences 


1 £. C\C\C\ 

16,000 


50.000 


lo. lo 


1 <c 
lo 


1 c t\C\C\ 

15,000 


A£L 0*7< 

4o.o /5 


lo: 15 


1 £L 

lo 


1 A (\f\(\ 

14,000 


43. /50 


i i a 
10. 14 


1 0 


13,000 


40.625 


16:13 


16 


12,000 


47.500 


16:12 


16 


11,000 


34.375 


16:11 


16 


10,000 


31.250 


16:10 


16 


9,000 


28.125 


16:9 


16 


8,000 


25.000 


16:8 


16 



TABLE 2 

. 3 

... 

s is- 

[0062| The method of the present invention can be used to replace a group of 

input pixels formed by the imager with a smaller group of output pixels. For 
example, Table 2 illustrates the fact that at an altitude of 10,000 feet, 16 input 
pixels obtained by the imager may be replaced with just 1 0 pixels interpolated in 
accordance with the present invention. This feature results in a reduced data 
transmission rate (reduced bandwidth) requirement for imaging systems that must 
transmit image data to a remote receiving station for storage and processing 
because fewer pixels must be transmitted in a given amount of time. This feature 
also reduces the amount of memory needed to store an image. How to interpolate 
pixels according to the present invention is further described below. 

[0063] Before describing how to interpolate pixels according to the present 

invention, however, it should be noted here that the altitude (frequency) 
subdivisions listed in Table 2 are not the only subdivisions possible according to 
the present invention. The equations described below allow for both coarser or 
finer subdivisions. For example, 500 feet subdivisions are possible when using 
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ratios of 32:X rather than 16:X. Similarly, 2,000 feet subdivisions are possible 
when using ratios of 8:X. As would be apparent to a person skilled in the relevant 
art given the discussion herein, other subdivisions are also possible. The present 
invention is not limited to only the subdivisions listed in Table 2. 

[0064] In general, pixel interpolation according to the present invention is a 

process by which the signal strength of a converted pixel is determined. The 
signal strength of a converted pixel depends on the signal strength of each of the 
input pixels received in step 202. The number of input pixels used in the method 
of the invention generally depends on the particular interpolation algorithm used. 

[0065] The amplitudes or intensities of the output pixels formed in step 208 can 

be derived using an interpolation formula of the form: 

z(kAx 2 )= X yinAx^ rX^k) EQ 2 

n 

where the sum, in general, is over the index "n" from n = -°° to n = +~. Here, z(k 
Ax 2 ) equals the amplitude or intensity of the kth output pixel, y(n Ax,) equals the 
amplitude of the nth input pixel, and r^n, k) equals an interpolating function 
centered on the output pixel. As shown in FIG. 4, Axi equals the sampling 
distance between two adjacent input pixels, and Ax 2 equals the sampling distance 
between two interpolated or output pixels. 
[0066] FIG. 5 illustrates one system that can be used to implement EQ. 2, and 

form the z output pixels from the y input pixels. The weighting factors or 
weighting cells (W^) shown in FIG. 5 are represented in EQ. 2 by the r^s of 
EQ. 2. 

[0067] There are several possible interpolating functions that can be used in 

accordance with the present invention to select the weighting factors used to form 
intermediate intensity values from the input pixels. For example, a cubic B-spline 
function can be used, as further described below. Other possible interpolation 
functions that can be used to select weighting factors include an n^-order spline 
function or a sine function. In fact, there are numerous references to interpolation 
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functions and algorithms that can be used in accordance with the present 
invention. SeeGERALDC.HoLST, Sampling, Aliasing, and Data Fidelity for 
Electronic Imaging Systems, Communications, and Data Acquisition 
144-48 (JDC Publishing, Winter Park, FL (1998)) (SPIE Press Series, No. 55). 

[0068] Using a cubic B-spline interpolation function, the method of the invention 

can be implemented using just the two nearest neighbors on either side of an 
interpolated pixel. Other interpolation functions can be used, and the present 
invention is not limited to using a cubic B-spline interpolation function. Other 
interpolation functions may require using a different number of input pixels to 
form an output pixel. 

[0069] A cubic B-spline function is defined as: 



1 2 
r 3 (k) - — • |z| 3 - z 2 ' + — , where 0 < |z| < 1 

= 7 • (2 - |z| 3 ), where 1 < |z| < 2 EQ. 3 

6 

= 0 elsewhere 



where: z = (nm s - k)/m s and m s is the spacing between input pixels. A cubic B- 
spline interpolation function has continuous first and second derivatives. 
[0070] This choice for r 3 (k, n) yields an interpolation formula of the form: 

Z(k AX 2 ) = X (" A *l ) • r 3 ("» *) E Q. 4 

n 

where the sum extends over the two nearest neighbors on either side of the 
interpolation point r 3 (k). It should be noted that the "z" variable used in EQ. 4 is 
not the same as the "z" variable used in EQ. 3. FIG. 6 illustrates graphically the 
evaluation of z(k Ax 2 ). 
[0071] Using EQ. 4 for each altitude range, the output pixels formed in step 208 

can be determined in terms of the initial intensity values (amplitudes) of the input 
pixels and the location of the new output pixels relative to the input pixels. Each 
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contiguous group of 16 input pixels is replaced by a group of output pixels 
illustrated in Table 2 above. For example, from Table 2, at 14,000 feet, the 
number of output pixels formed from 16 input pixels is 14. 
[0072] Table 3 below shows the intensity values of the 14 output pixels formed 

from 16 input pixels obtained by sampling the sinusoidal pattern of EQ. 5. 

( 27TX n ^\ 

y= 1+sin^— j EQ.5 



where X equals 250 /^m, and x n equals the input pixel sampling points {0 ^m, 10 
//m, 20 //m, . . . }. FIG. 7 shows a graph of the waveform, the pixel sampling 
points, and the output sampled values (output pixels) formed in accordance with 
the method of the invention. 

[0073] In optional step 210, the number of output pixels formed in step 208 is 

selected based on a distance between the imager and an object. This feature of the 
invention is illustrated, for example, in Table 2. As illustrated in Table 2 and 
described herein, sixteen input pixels can be replaced by fourteen output pixels 
when the imaging device is carried aboard aircraft 100 flying at an altitude of 
14,000 feet (i.e., the distance between the imager and objects on the ground is 
about 14,000 feet). As also illustrated in Table 2, sixteen input pixels can be 
replaced by nine output pixels when the imaging device is carried aboard aircraft 
100 flying at an altitude of 9,000 feet (i.e., the distance between the imager and 
objects on the ground is about 9,000 feet). This feature of the invention can be 
implemented, for example, by setting switches (not shown) of system 500 that 
determine how many output pixels will be used to replace a predetermined number 
of input pixels. If it is known that aircraft 100 will be flying at an altitude of 
12,000 while the imager is operational, the switches can be set (e.g., before 
aircraft 100 takes offs for its mission) so that twelve output pixels are formed 
from a group of sixteen input pixels. 

[0074] In optional step 212, the number of output pixels formed in step 208 is 

dynamically adjusted based on a distance between the imager and an object. 
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According to this feature, if aircraft 100 is flying at an altitude of 14,000 feet 
when the imaging device is activated by the pilot, fourteen output pixels would be 
formed from sixteen input pixels. Similarly, if aircraft 100 is flying at an altitude 
of 8,000 feet when the imaging device is activated by the pilot, eight output pixels 
would be formed from sixteen input pixels. An advantage of this feature is that 
there is no need to set switches before aircraft 100 takes off for its mission. The 
number of output pixels formed can be based, for example, on an input taken from 
an altimeter onboard aircraft 100. Thus, if aircraft 100 has to fly at an altitude 
other than that planned for prior to the takeoff of aircraft 100, this feature of the 
invention will nevertheless ensure an optimal number of output pixels is formed 
from a predetermined number of input pixels. This feature allows different legs 
of a tactical reconnaissance imaging mission to be flown at different altitudes 
without affecting image resolution. 

[0075] The steps of the method described herein can also be carried out in two 

dimensions (e.g., when used with a staring imager) provided an imager has excess 
resolution in two dimensions. 

[0076] In hyperspectral imaging, in which one axis is wavelength and the other 

axis is spatial distance, the spatial dimension is amenable to processing. In this 
case, the one-dimensional embodiment of the present invention described herein 
can be applied to the spatial dimension, while the spectral dimension is maintained 
unchanged. 

[0077] In carrying out the method of the present invention, as described herein, 

each group of 16 input pixels is operated upon to produce a group of output 
pixels, which comprises 16 or fewer pixels. In order to better understand this 
aspect of the method, imagine that a first group of twenty input pixels is received 
and temporarily stored in a buffer 8 1 0, as shown in FIG. 8 A, to form a first group 
of output pixels (not shown). Buffer 8 1 0 is similar to buffer 5 1 0 in FIG. 5 except 
that buffer 810 comprises twenty memory locations, or cells C 0 -C 19 , instead of 
five. As shown in FIG. 8A, the first group of twenty input pixels comprises a 
group of sixteen input pixels {Y 2 , Y 3 . . ., Y 16 , Y 17 } to be replaced by a group of 
output pixels, plus four additional pixels {Y 0 , Y,, Y 18 , Y J9 } (i.e., two pixels on 
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each end of the group of sixteen input pixels) to be used in forming the group of 
output pixels. As illustrated in FIG. 8 A, pixel Y I8 is in cell C 18 of buffer 810 and 
pixel Y )9 is in cell C 19 . 



in buffer 810 so that another next group of sixteen input pixels to be replaced is 
located in cells C 2 through C, 7 , as shown in FIG. 8B. These pixels are shifted into 
buffer 810 from a stream of input pixels (not shown). As illustrated in FIG. 8B, 
pixel Y 18 is stored in cell C 2 of buffer 810 and pixel Y 19 is stored in cell C 3 after the 
shift. Pixels Y 16 and Y 17 occupy cells C 0 and C 1? respectively, after the shift, as 
shown in FIG. 8B. Pixels Y 16 and Y 17 are used in forming a second group of 
output pixels, as shown in FIG. 8B, even though they were also used to form the 
first group of output pixels. After the second group of output pixels is formed, 
new input pixels are shifted into buffer 810 again, as illustrated in FIG. 8C. This 
process continues until all the input pixels of the input pixel stream have been 
converted. 

[0079] One advantage of the method, as described herein, is that the output group 

of pixels for each input group of pixels can be determined in the same manner, 
independent of which input group is being processed. Thus, the weighting factors 
described herein can be predetermined, and each output pixel can be formed by 
summing the weighted amplitudes of neighboring input pixels (e.g., intermediate 
intensity values). This method of forming output pixels from input pixels is 
simpler to implement than methods that determine weighting factors on the fly. 

[0080] As described herein, the method of the invention can be used to improve 

the signal to noise ratio (SNR) of a stream of pixels. The SNR is improved for the 
same statistical reasons that the variation of the mean of a number of samples has 
a smaller standard deviation than the samples themselves. This feature of the 
invention is shown by the following set of calculations. 

[0081 ] In an embodiment of the invention, the signal and noise associated with an 

interpolated signal are given by EQ. 6 and EQ. 7. 



[0078] 



After the first group of output pixels is formed, the input pixels are shifted 



4 




EQ. 6 
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Noise (mts) = 




EQ. 7 



where r t ^0 

EQ. 6 is similar to EQ. 2 and EQ. 4. In EQ.6, the variables have been rewritten 
using a form that will clarify the following description of the invention. The four 
Sj's in EQ. 6 represent the amplitudes of four original (input) pixels surrounding 
the location of an interpolated (output) pixel. The r/s in EQ. 6 represent, for 
example, the cubic splines represented by r 3 in EQ. 4. In EQ. 7, the four N/s 
represent the noise associated with each of the four original pixels. The noise 
associated with each of the four original pixels may be assumed to be 
uncorrected . 

[0082] For a case in which the waveshape to be interpolated according to the 

method of the invention is a constant, each Sj in EQ. 6 is equal to a constant "A." 
It follows, therefore, that the interpolated signal must also be equal to the same 
constant "A," as illustrated by EQ. 8. 

4 4 

From EQ. 8, it follows that the sum of the r^s equals one, as shown in EQ. 9. 
4 

/=i 

[0083] As will be understood by a person skilled in the relevant art, EQ. 8 and 

EQ. 9 may also be used where the waveshape to be interpolated is not constant, 
but varies slowly between pixels, e.g., in small neighborhoods where the S/s and 
N/s are approximately constant. This point is demonstrated by the data in Tables 
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3-5 below. In Table 4, the r/s for a particular implementation of the invention 
have been calculated. As can be seen by looking at the data in Table 4, each set 
of Tj's obeys EQ. 9. 

[0084] Using EQ. 9, it will now be demonstrated that the method of the invention 

can be used to improve the signal to noise ratio (SNR) of a stream of pixels. 
Using EQ. 6 and EQ. 7, the SNR of an interpolated signal is given by EQ. 10. 



SNR . ' 10 



As would be known to a person skilled in the relevant art, a sum of squares is 
always less than or equal to the square of the surri, as illustrated by EQ. 1 1 and 
EQ. 12. 



where N ; , Nj, r f , and ij are all positive numbers. 

( ^ 2 

.Ufa • r i) 2 * 5>i-i 



EQ. 12 



Using substitution, EQ. 10 can be rewritten as EQ. 13. 



SNR = 



Xs ( .r, 



EQ. 13 
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[0085) Applying the principle of EQ. 9 to EQ. 6 and EQ. 7, and assuming the S/s 

and N/s are constant within a small region, produces EQ. 14 and EQ. 15. 

2>, •/•,="$ = $ EQ. 14 

]T TV,. •r i = N* = N E Q- 15 



Finally, substituting EQ. 14 and EQ. 15 into EQ. 13 produces EQ. 16, which 
demonstrates that the SNR of the interpolated signal is greater than or equal to the 
SNR of a single pixel in the neighborhood of interest. 



s 

SNR >— EQ. 16 

TV 



(0086) To further illustrate this feature of the invention, a comparison of the 

average SNR of interpolated signals to original signals was performed. The 
results of this comparison are presented in Table 3 and Table 5. As can be seen, 
the comparison shows that the invention can be used to improve the SNR of a 
stream of pixels. 

[0087] The test signal used for the comparison is given in EQ. 1 7. 



1 + sin 



V 



2 ™n 11 



where: X equals the test signal wavelength; 

x n equals the original sampling points {0 urn, 10 um, 20 jim, ...} , and N e 
is the number of electrons generated in a pixel by the incident photons 
from the scene being imaged. 
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A value of 500 for N e was selected for computational ease in the comparison. In 
implementations of the invention, a typical value for N e is 1 x 10 6 electrons. In 
the generation of Table 3 and Table 5, the noise is assumed to have a Poisson 
distribution so that N = S' /2 . 



allows margin for the Nyquist limit of two pixels per sample for the interpolated 
samples at a separation of 40 (am (8,000 foot altitude case). The original 16 
samples for the test signal were replaced in two separate runs by 14, and 8 
samples, corresponding to aircraft altitudes of 14,000 feet and 8,000 feet, 
respectively. The results of the runs are shown in Table 3 and Table 5. 

[00891 The data in Table 3 and Table 5 illustrate extremes in the number of 

interpolated (output) pixels. At an altitude of 14,000 feet, sixteen original (input) 
pixels are replaced by 14 pixels. At an altitude of 8,000 feet, sixteen pixels are 
replaced by eight pixels. 

[0090] Generally, as can be seen in the data in Table 3 and Table 5, the 

improvement in average SNR obtained using the invention is independent of 
aircraft altitude and/or the number of output pixels. The variation seen in the data 
of Table 3 and Table 5 is primarily a result of the averaging scheme used in the 
example comparison. The average SNR of the interpolated samples is seen to be 
greater than the average SNR of the original samples in both tables. 

[0091] In the limit of a constant amplitude signal at a target, the SNR can be 

calculated analytically for an aircraft flying at an altitude of 8,000 feet. Table 4 
shows the values for the r ; coefficients, which are the same for all pixels. The 
SNR is given in EQ. 18. 



[0088] 



A value of 100 um was chosen for X for the first set of tests. This value 





SNR = 



EQ. 18 



In this example the noise is assumed to have a Poisson distribution so that N = S' A . 
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Aircraft Altitude = 14,000 ft 


Original Samples 


Interpolated Samples 


index 


distance 
fim 


signal 
electrons 


S/N 


index 


distance 


signal 
electrons 


noise 


S/N 


1 


0 


500 


22.22 


1 


0.0 


500 


22.4 


22.22 


2 


20 


976 


32.15 


2 


22.9 


880 


21.5 


41.667 


3 


40 


794 


28.57 


3 


45.7 


603 


17.6 


34.483 


4 


60 


206 


14.286 


4 


68.6 


150 


7.9 


18.868 


5 


80 


24 


4.95 


5 


91.4 


305 


11.9 


25.641 


6 


100 


500 


22.22 


6 


114.3 


800 


20.2 


40.00 


7 


120 


976 


31.25 


7 


137.1 


777 


19.9 


38.462 


8 


140 


794 


28.57 


8 


160.0 


274 


10.7 


25.641 


9 


160 


206 


14.286 


9 


182.9 


161 


6.6 


24.39 


10 


180 


24 


4.95 


10 


205.7 


633 


17.0 


37.037 


11 


200 


500 


22.22 


11 


228.6 


872 


20.4 


43.478 


12 


220 


976 


31.25 


12 


251.4 


465 


14.3 


32.258 


13 


240 


794 


28.57 


13 


274.3 


118 


5.8 


20.00 


14 


260 


206 


14.286 


14 


297.1 


432 


14.9 


29.412 


15 


280 


24 


4.95 








avg. 


31.00 


16 


300 


500 


22.22 
















avg. 


20.38 













TABLE 3 



Ri Coefficients 
( 8,000 Feet Altitude Exam 


pie) 


r -2 


0.1667 




0.0278 


f -l 


0.6667 




0.0444 


r l 


0.1667 




0.0278 


r 2 


0 




0 




1.000 




0.1000 



TABLE 4 
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Aircraft Altitude = 8,000 ft 


Original Samples 


Interpolated Samples 


inde 

X 


distance, 
um 


signal, 
electrons 


S/N 


inde 

X 


distance, 
jim 


signal, 
electron 
s 


noise 


S/N 


1 


0 


500 


22.2 


1 


0 


500 


22.4 


22.22 


2 


20 


976 


31.25 


2 


40 


726 


19.6 


37.04 


3 


40 


794 


28.57 


3 


80 


134 


5.5 


14.39 


4 


60 


206 


14.286 


4 


120 


866 


21.7 


40.0 


5 


80 


24 


4.95 


5 


160 


274 


10.7 


25.64 


6 


100 


500 


22.22 


6 


200 


500 


15.8 


31.25 


7 


120 


976 


31.25 


7 


240 


726 


19.6 


37.04 


8 


140 


794 


28.57 


8 


280 


134 


5.5 


24.39 


9 


160 


206 


14.286 








avg. 


29.00 


10 


180 


24 


4.95 












11 


200 


500 


22.22 












12 


220 


976 


31.25 












13 


240 


794 


28.57 












14 


260 


206 


14.286 












15 


280 


24 


4.95 












16 


300 


500 


22.22 
















avg. 


20.38 













TlJ TABLE 5 



Method for Transforming Image Resolution 

[0092] As described herein, the present invention is particularly well adapted for 

use with an airborne imaging system. A further feature of the invention allows a 
variable resolution image to be converted to a fixed resolution image. This feature 
of the invention can be used, for example, to control access to an image database. 

[0093J FIG. 9 is a flowchart of a method 900 for transforming image resolution 

according to the invention. As illustrated in FIG. 9, method 900 comprises five 
steps. 

[0094] In step 910, an image having an initial resolution is retrieved from a 

memory. The image retrieved comprises at least a first and second original pixel 
having an initial intensity value. As would be known to a person skilled in the 
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relevant art, an image can be stored, for example, on a computer or a server that 
forms part of an interconnected network of computers. Images are typically 
stored as an array of digital values. Storage mediums that are capable of storing 
these digital values are typically referred to as memory. Groups of images can be 
stored together in memory as a database of images. Storing images in a database 
allows a requester located at a network terminal to request and receive images 
stored in memory. 

[0095] As described herein with regard to the method of the invention, the pixels 

of the image retrieved in step 910 can be optionally filtered in order to prevent 
aliasing. There are many lowpass filters and/or filtering techniques that can be 
applied to the pixels retrieved in step 9 1 0 in order to prevent aliasing. Given the 
description of the present invention herein, a person skilled in the relevant art will 
be able to determine whether the pixels of the image retrieved in step 910 should 
be lowpass filtered. 

[0096] . Steps 920, 930, and 940 of the method 900 are described together. In . 

steps 920 and 930, at least one intermediate intensity value for each pixel of the 
image retrieved in step 910 is formed. In step 940, the intermediate intensity 
values formed in steps 920 and 930 are combined to form a group of output 
pixels. How to implement steps 920, 930, and 940 will be known to a person 
skilled in the relevant art given the description of the invention herein. 

[0097] In step 950 of method 900, steps 920, 930, and 940 are repeated for 

additional pixels of the image retrieved in step 910. Step 950 ends when a copy 
of the image retrieved in step 910 has been formed. The copy of the image 
formed has a resolution that is different than the resolution of the image retrieved 
in step 910. 

[0098] Method 900 can be used, for example, to control access to a database of 

images. For example, several images having a resolution of one-meter may be 
stored in a commercial database of images, which are available for purchase by 
members of the public (i.e, image requesters) via the INTERNET. For business 
reasons, the owner of the database may want to charge requesters who access the 
database a price that varies depending on the resolution of an image retrieved 
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from the database. For example, the owner of the database may charge one price 
for retrieving a one-meter resolution image, and a second, lower price for 
retrieving a three-meter resolution image. 

[0099] Using the invention described herein, the owner of the database can store 

a one-meter image in an image database, and use method 900 to generate a copy 
of the image having a three-meter resolution for sale to a requester who only 
needs and wants to pay for accessing a three-meter resolution image. A software 
header attached to a request to access an image can be used, for example, to 
determine the resolution of the copy of the image produced in step 950 of method 
900. Having an ability to make copies of an image stored on a server (a variable 
resolution image) with any desired resolution (a fixed resolution image) eliminates 
the need for storing multiple copies of an image, each image having a different 
resolution, on the server. Thus, the invention can be used to reduce costs to the 
owner of an image database. 

[0100] Additional features and applications of method 900, as well as how to 

implement these features and applications, will be apparent to a person skilled in 
the relevant art given the description of the invention herein. 

Conclusion 

[0101 ] Various embodiments of the present invention have been described above. 

It should be understood that these embodiments have been presented by way of 
example only, and not limitation. It will be understood by those skilled in the 
relevant art that various changes in form and details of the embodiments described 
above may be made without departing from the spirit and scope of the present 
invention as defined in the claims. Thus, the breadth and scope of the present 
invention should not be limited by any of the above-described exemplary 
embodiments, but should be defined only in accordance with the following claims 
and their equivalents. 



